Technical Field
The invention relates to methods and apparatuses for image reconstruction. The invention relates in particular to methods and apparatuses in which an image is reconstructed mathematically from a plurality of single images showing the same object.
Background Information
Modern optical systems are nowadays to satisfy ever higher demands in terms of image quality. This can be attained by means of high-precision and high-quality lenses (for example in the reflex camera field). In alternative approaches, less expensive lenses are used in combination with subsequent post-processing. The latter approaches are used, for example, in cameras integrated in mobile telephones. Although high-quality lenses exhibit a better image quality, they are substantially more expensive in terms of cost price. However, the approach of using cheaper lenses with subsequent post-processing also has limitations, because with conventional approaches optical aberrations can often be compensated for only with losses in image quality. How effectively optical aberrations can be deconvolved depends substantially on the contrast of the modulation transfer function (MTF) and on the signal-to-noise ratio (SNR) of the image. If at some spatial frequencies the contrast given by the MTF in relation to the SNR is low, those spatial frequencies cannot be reconstructed or can be reconstructed only under certain additional assumptions. This is the case in particular for spatial frequency ranges in which the MTF has zeros due to aberration. Such zeros in the MTF can occur, for example, in the case of astigmatism or defocus.
In addition, wavelength-dependent optical aberrations can also occur with simpler lenses. Examples thereof are transverse chromatic aberrations and longitudinal chromatic aberrations, which cannot be compensated for, or can be compensated for only with difficulty, by conventional approaches to digital post-processing.
The following approaches have conventionally been followed in order to achieve a higher image quality:
Higher-quality lenses can be used. However, as well as having cost disadvantages, this can also lead to a large installation space and high weight. In addition, with a larger number of lenses, the susceptibility of the system to reflection can be increased and/or the transmission can be reduced. This is disadvantageous for many applications in the end user field, but also in the field of expensive special equipment.
Simpler lenses can be used and deconvolution methods can be employed. Optical aberrations caused by less expensive lenses can be determined by simulations or measurements after manufacture and then deconvolved with the aid of known deconvolution methods, for example the Wiener filter. However, aberrations can be compensated for to only a certain degree using such techniques. In particular when the contrasts of the MTF in relation to the noise are small, various image artefacts, for example image noise or so-called ringing artefacts, can form depending on the deconvolution method.
When the sensitivity of an image sensor covers a relatively large wavelength range, the point spread function (PSF), or MTF, can change wavelength-dependently. This is the case, for example, when each of a plurality of color channels of the image sensor covers a relatively large wavelength interval. This makes image reconstruction more difficult, because a convolution with different PSFs takes place wavelength-dependently. A deconvolution can be made, for example, with a polychromatic PSF, with which the variation of the PSF with the wavelength can be taken into consideration at least to a certain degree. However, digital compensation of transverse chromatic aberrations, for example, continues to be possible to only a limited extent. With polychromatic PSFs, it is only possible to compensate for the shift between two color channels with conventional approaches. The transverse chromatic aberrations within the sensitivity range of a color channel that are not compensated for persist at least in part. A limitation to the effect that the PSF for a subsequent image reconstruction is not to change or is to change only slightly within the spectral range of a color channel represents a great limitation for the lens design and requires more expensive and more accurate lenses. In addition, with conventional approaches, a wavelength-resolved reconstruction is possible to only a limited degree and is frequently limited to the number of color channels of the image sensor.
In order to make improvements in respect of the mentioned disadvantages, reconstruction methods can use so-called image priors. This is described, for example, in J.-H. Lee, Y.-S. Ho. Non-blind image deconvolution with adaptive regularization. Advances in Multimedia Information Processing—PCM 2010, Lecture Notes in Computer Science, 2010, Volume 6297/2010, pages 719-730, Springer-Verlag Berlin Heidelberg 2010. Methods using image priors attempt to model typical properties of a good, that is to say sharp and noise-free, image. The image priors can be used in the deconvolution as regularisation and “penalise” results with pronounced artefacts. Image priors are, however, only a limited modelling of the perfect image, because the properties of a good image are also dependent on the object. Therefore, reconstruction using image priors can result in blurred reconstructed images. The quality of the resulting image can be of greatly differing quality depending on the object.
Coded apertures or phase masks, for example, can be used to produce images with extended depth of field. Aberrations are introduced which have the property of being readily deconvolvable, other aberrations (such as, for example, a defocus) being lost therein. After deconvolution, an image with a high depth of field is obtained. The optimum form of the coded aperture depends on the aberrations, so that it must be specially optimized and manufactured for each optical system. A specific coded aperture is frequently optimum in the corresponding optical system only for a very specific zoom and focus setting.
In addition, there are sharpening methods which require no or only limited information about the optical system. WO 05/031645 A, for example, describes a method in which a less sharp image is produced mathematically from an image taken in focus and those images are then combined mathematically, no information about the optical system being required. In methods that do not take optical imaging properties into consideration, the resulting image quality can be limited compared to deconvolution methods. Noise can be marked. Pronounced aberrations cannot be compensated for by methods that do not use information about the optical system.